Papers with pre-trained vision and
CPL: Counterfactual Prompt Learning for Vision and Language Models (2022.emnlp-main)
Copied to clipboard
Xuehai He, Diji Yang, Weixi Feng, Tsu-Jui Fu, Arjun Akula, Varun Jampani, Pradyumna Narayana, Sugato Basu, William Yang Wang, Xin Wang
| Challenge: | Existing prompt tuning methods tend to learn spurious or entangled representations, leading to poor generalization to unseen concepts. |
| Approach: | They propose a prompt tuning technique that tunes the learnable prompt for pre-trained vision and language models. |
| Outcome: | The proposed method improves few-shot performance on vision and language tasks over existing prompt tuning methods. |
Lexi: Self-Supervised Learning of the UI Language (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Existing models rely on UI metadata, which is often missing or not accessible. |
| Approach: | They propose a vision and language model that can learn generic visio-linguistic representations of UIs . they use a dataset to train Lexi, which is based on UI metadata . |
| Outcome: | The proposed model can handle unique features of UI screens, including text richness and context sensitivity. |